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1 . INTRODUCTION 



This report considers a problem of selecting from a set 
of objects a subset of maximum value where there are sequence 
dependent constraints which restrict the choices available. 

There is a natural ordering among the objects which dictates 
the order in which they must be considered. It is assumed 
that the decision maker has complete knowledge of the opportu- 
nities which will arise so that the problem is one of determining 
which opportunities to select and which to forego in preparation 
for later gain. The problem in which the opportunities have 
some probability of vanishing without notice is considered 
briefly, but the solution method is the same as long as these 
probabilities are known. 

The problem can be cast in several forms. Reference 1 
describes the problem in the context of an "investigator" pass- 
ing through a region in which there is a number of points to 
be examined. It can also be viewed as a "delivery" problem 
in which a delivery truck has a list of locations and delivery 
times. If a delivery is made, it must be done at the given 
time and the specified location. Knowing the transit times 
from point to point and the value of each customer, the problem 
is to select the most valuable subset of customers to serve. 

The problem can be viewed as a job shop scheduling 
problem in which the production manager has a set of possible 
jobs to perform. Each job has a required starting time and 
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known set-up and processing time. The problem is to select 
the most valuable subset for processing. Reference 2 deals 
with the application of dynamic programming to a similar problem 
in which each job has an availability interval within which it 
must be processed if selected. 

The problem is viewed here as one in which a defender is 
guarding some region against attack from a set of attackers 
whose times and points of arrival at the region are known. The 
defender is constrained to remain on the boundary of the region 
and must move to the point of intrusion at the time the attacker 
arrives in order to destroy the attacker. This is equivalent 
to saying that the defender can not engage the target until it 
crosses some threshold but it must be engaged immediately there- 
after. The selection problem arises because the defender is 
limited in the rate at which he can move along the boundary. 

For convenience the boundary is assumed to be a straight 

line segment along which the defender moves. Other configuration 
are possible. For example, some problems in point defense can 
be formulated in the same way. In that case the boundary can 
be thought of as a circle (of zero radius) and the position of 
the defensive system is represented by its angular rotation 
or orientation. Figure 1 illustrates a problem of this type 
when the region defined is the portion of the x-axis from 
0 to B. 
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FIGURE 1. Illustration of Basic Problem 
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It is convenient to think of all the attackers as 



having the same speed and travelling on parallel courses 
directly toward the boundary, but it is not necessary to do so. 

In fact, all that is required is the time and point of arrival 
on the boundary for each attacker. It is assumed hereafter 
that the attackers are numbered in order of arrival at the 
boundary and that the times and points of arrival are known. 

Let t. be the time of arrival of attacker j and p. 

J J 

be the point of arrival where 0 <_ p_. <_ B. The value of 
attacker j is represented by v^ . It is also convenient 
to define a dummy point Pg , tg with value Vg = 0 representing 
the initial location of the defender. 

For visual presentation of the problem a slight trans- 
formation is convenient. This problem is equivalent to one in 
which the attackers are stationary and the boundary moves in 
time through the attackers. As an example, the defender's 
position in the two-dimensional space (x,t) is shown in 
Figure 2. 

The problem remains one of determining the optimal set 
of targets for the defenders to engage subject to limitations 
on his ability to reposition his defensive system. 

This report deals primarily with the problem in which 
there are two identical defenders although the ideas are 
applicable for the M-de fender problem. A straightforward 
dynamic programming approach for the M-defender problem is 
discussed first. See also Reference 1. Next an efficient 
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FIGURE 2. Space-Time Representation 
of Defender's Position 
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solution method for the problem with a single defender is 
considered. Then an improved formulation for the two-defender 
problem is presented and it is shown how that problem can be 
solved with a greatly reduced computational burden. A computer 
problem was written to test the method and the solution to a 
sample problem with N = 50 attackers is included. 

Several generalizations are discussed in Section 5. 



2. DYNAMIC PROGRAMMING SOLUTION FOR M-DEFENDERS 

The M investigator problem can be solved with a 
standard dynamic programming approach using M state variables. 
Define the stages such that stage n corresponds to the 
time t n , and let X n = (x^ n , . . . , x^) be the state variable 
describing the location along the boundary of each investigator 
at stage n. The return function for stage n is 



r (X ) 
n n 



v x . — p / any i 

n m ^ n 



0 otherwise. 



The stage transformation functions can be written as 



x .. = t (x , D ) 
n+± n n n 



where D n is the vector of decisions prescribing the heading 
of all investigators as they move from stage n to n+1 and 
t is the function which describes the resulting locations 
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at stage n+1. The decisions D are restricted to lie in 

n 

some set K . An alternative formulation simply defines 5 
n n 

as the set of points to which the investigators move so that 

x , = t ( x , D ) = 5 , but still 5 is constrained to lie 
n+i n n n n n 

in some set of feasible solutions, say K . 

1 n 

The complete problem can be written as 



maximize 




subject to 



x 



n+1 



t 

n 





D 

n 



C K n 



n = 0 , . . . , N 

n = 0 , . . . , N 



and can be solved using a standard dynamic programming approach 
with tabular computations once a suitable grid has been 
established for the state and decision variables. The solution 
requires the evaluation of the following recursive equations 
beginning with stage N 

f N+l (x N+i = 0 



f (x ) = max {r (x ,5 ) + f , (x , )}, n=0,...,N, 

nn ^pr nnn n-ri n+1 

n w n 



where 



x , . = t (x , D ) . 
n+1 nnn 
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This method is very general and can be used to solve 
several variations of the basic problem, except that it 
becomes computationally unwieldy as M increases. The amount 
of computation required rises exponentially with M but 
linearly in N. 

Two generalizations that can be solved using a similar 
formulation are the problems in which there is a restriction 
for each defender on the total number of attackers it can 
engage or a restriction on the total lateral movement of 
the defender. In these problems the vector of state variables 
must be enlarged to include a component for the number of 
engagements remaining and a component for the amount of lateral 
movement remaining. 

The formulation above is not practical for large 
values of M nor for problems with constraints which lead 
to additional state variables, and the purpose of this report 
is to present a method which is more efficient than that just 
given, but first an efficient method for the basic one-state- 
variable problem is presented. 
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3. EFFICIENT SOLUTION FOR ONE DEFENDER 

For the problem with a single defender, as illustrated 
in Figure 1, the algorithm shown in Figure 3 provides an 
efficient solution procedure. That algorithm assumes a dummy 
point at Pq having value Vq = 0 representing the starting 
location of the defender. The computation produces in reverse 
order the sequence of optimal return functions f , n = 0,...,N 
where the quantity f represents the optimal total return 
that can be obtained from the remaining stages, not including 
n, given that the position of the defender is p n at time t^ . 
The sequence of optimal decisions is also produced and can be 
used beginning at d^ to trace the optimal policy. The 
algorithm relies on the fact that the only relevant value of 
the state variable at any stage is the value corresponding to 
the location of the attacker and that the return functions 
need not be computed for any other values. For that reason 
the state variable is suppressed in f . 

This algorithm can easily be implemented in FORTRAN as 
two nested DO loops . 



4. IMPROVED FORMULATION FOR THE TWO-DEFENDER PROBLEM 

This section shows how an improved formulation for the 
problem with two defenders can be obtained. In this section the 
N— attacker problem is still viewed as an N stage dynamic 
programming problem where stage n corresponds to the time t n 
at which the defender departs from the target n. In this 
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FIGURE 3. Algorithm for the problem with a single defender 
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formulation, as in the formulation of Section 3, passing through 
stage n corresponds to engaging target n; and if target n is 
not engaged, then stage n is bypassed. 

At stage n the state variables x n ^» x n2 re P re sent 
the positions of defenders 1 and 2 along the boundary. For 
computations it is assumed that a grid is established for the 
state variables. For convenience this grid is assumed to be the 
same at each stage, consisting of S+l points equally spaced a 
distance A apart. It is also convenient to assume that the 
targets occur at the grid points, although this is not essential. 
If they were located between grid points it could be assumed that 
a target is engaged if the defender passes within a distance r 
of the target, or the grid points could be redefined at each 
stage to include the target location. 

One observation which considerably reduces the amount 

of computation is that if the defenders pass through stage n, then 

one of the defenders must be located at p , the location of the 

L n 

til 2 

n attacker. Thus instead of (S+l) possible values of 

the state variables x n ^ and we need to consider only 

(S+l) + (S+l) - 1 = 2S + 1 values. The computational reduction 

from this observation is roughly equivalent to reducing the 

number of state variables by one. 

A second observation can also be made. This is that 
there is an optimal solution in which the two defenders do not 
cross. This is similar to the observation in the travelling 
salesman problem that for problems with Euclidean distance 
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the optimal tour need not cross itself. The validity of this 
observation can be established by noting that if a point of 
crossing occurs, the defenders could simply be renumbered 
so that number 1 remains on the left and number 2 on the right. 
This follows from the fact that the defenders are identical, 
and it would not hold if they were able to move at different 
rates along the boundary. It would also be invalid if a penalty 
were assessed for changing the direction of motion from left 
to right or vice-versa or if they had constraints on the number of 
attackers handled. 

With this second observation the number of possible 

values of the state variables is reduced even further. The 

possible values of the state variables at stage n, x ^ and 

x _, are given in Table 1 where p is the x-coordinate of 
nz u n 

■hH 

the n target. Since the targets were initially assumed to 
lie on the grid points p n = KA for some K. The state variable 
combinations listed in Table 1 have been numbered for reference, 
but notice that knowing the state number is equivalent to 
having the value of both state variables. For example, state 
number K' implies that 




K'A , if K ' A < p i 



K ' A > Pi 



and 




p . A , if K ' A < p . 



K'A > p. . 
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State 

Number 




X -i 

nl 


x n2 


0 




0 




1 




A 


Pn 


K-l 




(K-l) A 


Pn 


K 




?n 


(K+l ) A 


S 




Pn 


(S+l) A 



TABLE 1 

Possible values of the state variable 
in the two defender problem. 
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Thus all the state variable information is conveyed by the 

state numbers and it is unnecessary to write both state 

variables. For this reason it is adequate to let the single 

state variable at stage n be the state number S so that 

J n 

S = 0,1/2,..., S. The result of this second observation is to 
n 

reduce the number of state variable combinations at each stage 
from 2S+1 to S+l. 

In the dynamic programming formulation f (S ) denotes 
the maximum total return from the remaining stages n+l,...,N 
given that the defenders are at stage n in state S . The 
general recursion can be written as 



VV - 0 



f (S ) = max {v. + f~ (D )} , 

n n n 5 r, n 

o t D n n 

n n 



3 n > n / D n ^ K ^ (S n ) , n - 1,...,N-1 

n 

where 5 n is a decision variable denoting the stage to which 

the defender moves next and D is the decision variable which 

n 

determines the state number to which each defender moves at 

stage 6 n « The set of feasible states at stage given 

state S at stage n is denoted by K. (S ) . 

n * o n 

n 

From state S n it may not be possible to go to some 
stages and, among those that are possible, the question of which to 
enter must be resolved. The general computation proceeds as 
shown in Figure 4 beginning at stage N-l. Figure 4 assumes 
completion of the computation of f (S ) for n = N-l, N-2,.-..,i+ 
so that the next step is to evaluate f . ( •) - 

l 
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FIGURE 4 
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5. MODIFICATIONS TO THE BASIC PROBLEM 

A similar problem in which each target j has some 
probability 1 - p^ of vanishing before it is engaged can 
be solved by the same method. The objective becomes maximization 
of expected value and the computation is modified by diminishing 
the value of target j from Vj to Pj v j‘ After this change, 
the remainder of the solution method is the same. 

Several other modifications of the basic problem are of 
interest. These include the problem in which there is a 
limitation on the total motion of the defender as would be the 
case where fuel or some other resource is consumed. Another 
interesting problem arises when there is a constraint on the 
total number of attackers which can be engaged by each defender. 
Both of these modifications can be handled in the basic dynamic 
programming formulation by the addition of another state vari- 
able; but as mentioned previously large problems require that 
more efficient methods be found. 

In the computer program which implements this solution 
method for the two-defender problem a further savings is 
incorporated. If the difference between 6^ and i is 
large, or at least if the time difference between the stages is 
large, it is almost certainly possible to engage some target 
between i and 6^, say at stage j, and still move to any 
desired state at stage 5^. In such a case it is unnecessary 
for the recursive procedure to examine stages 6^, 6^ + -^,..., N 
as possible choices of the next stage following i. This is 
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because those choices are dominated by another choice, namely, 
moving to stage j from i. A test to this effect was incorporated 
with a considerable reduction in computer time. The usefulness 
of such a test depends on the particular rules of motion assumed 
for the defender and will be pursued here. 

Reference 3 considers the problem from the point of view 
of two attackers moving through a region containing N targets . 
Each attacker must select a subset of the targets and no target 
can be selected by both attackers. The case in which the 
attackers enter the region from different directions is considered. 
That case is not covered here since the ordering of the stages 
differs for the two attackers. 



6 . COMPUTER PROGRAM 

A program was written in FORTRAN IV to implement the 
procedure described in Section 4 for the two defender problem. 
This program was used to solve several problems in which the 
attacker's locations were generated randomly using a uniform 
distribution to determine the coordinates x and y, where 
0 <_ x < 11 and 0 £ t £ 100. In order to make the x-coordinates 
fall on a conveniently spaced grid, the x-values generated were 
truncated to an integer before solving the problem. 

Table 2 shows the x and t coordinates for a sample 
problem involving 50 attackers. Also shown are the randomly 
generated target values ranging from 1 to 5 . 
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A graphical presentation of the sample problem is 
given in Figure 5. The numbers beside each point are the 
target values on the right and the point number (mod 10) on 
the left. The two lines show the sets of attackers engaged 
by the two defenders. The roles of motion used in this sample 
problem permitted the defenders to move left or right at the 

y 

rate of one unit per unit time. The scale used in plotting 
Figure 5 is compressed in the t direction and it gives the 
appearance that the defenders can move more rapidly than one 
unit per unit time, but reference to the coordinates of each 
point will confirm that the rules of motion are not violated. 
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1 


5.00 


0.10 


1 


2 


6.00 


2.33 


4 


3 


6.00 


5.20 


1 


4 


4 .00 


5.53 


2 


5 


7.00 


7.74 


2 


6 


6.00 


8.96 


3 


7 


6.00 


9 .31 


1 


8 


4.00 


16.18 


3 


9 


3.00 


17.18 


4 


10 


4.00 


20.88 


4 


11 


2.00 


20.80 


5 


12 


6.00 


21.37 


1 


13 


4.00 


22.19 


4 


14 


9.00 


22.81 


5 


15 


1.00 


26.16 


2 


16 


9.00 


26.91 


3 


17 


6.00 


31.96 


1 


18 


1.00 


32.69 


5 


19 


7.00 


42.18 


4 


20 


7.00 


43.04 


3 


21 


3.00 


44.45 


2 


22 


4.00 


45 .69 


2 


23 


5.00 


49.25 


5 


24 


7.00 


50.33 


3 


25 


8.00 


53.22 


5 


26 


2.00 


55 .22 


2 


27 


7.00 


59.42 


1 


28 


3.00 


59.88 


5 


29 


2.00 


60 . 38 


3 


30 


9.00 


60.63 


4 • 


31 


8.00 


65.35 


2 


32 


6.00 


65.54 


5 


33 


8.00 


66.17 


3 


34 


3.00 


65 .79 


2 


35 


3.00 


66.89 


2 


36 


2.00 


69.05 


2 


37 


9.00 


70.86 


3 


38 


4.00 


72.45 


5 


39 


0.00 


73.54 


4 


40 


8.00 


75.19 


3 


41 


6.00 


76.55 


3 


42 


4.00 


76.92 


2 


43 


7.00 


80 .05 


2 


44 


0.00 


81.85 


2 


45 


4.00 


84.41 


1 


46 


9.00 


85.02 


3 


47 


6 .00 


90.00 


1 


48 


7.00 


90.34 


5 


49 


8.00 


92.90 


1 


50 


4.00 


99 .97 


5 



Table 2. Data for Sample Problem Involving 50 Attackers 
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